Telomere and telomerase are a key factors that regulate cell replicative lifespan. Telomere shortening has been observed in many types of cells in vitro and in cross-sectional analyses, and significantly shortened telomeres induce cell senescence and apoptosis. However, it remains to be determined how telomere length change in vivo and whether shortened telomeres contribute to age-associated decline of function. Telomerase is an enzyme that synthesizes telomere and is highly regulated in lymphocyte activation. To understand the in vivo change of telomere length and telomerase activity, we followed 200 participants (age range at the entry from 21 to 91) from the Baltimore Longitudinal Study of Aging (BLSA) over an average of five years and analyzed telomere length, telomerase activity, lymphocyte composition and health conditions in peripheral blood lymphocytes and monocytes at the beginning and five-year follow-up. We found that in PBMC, lymphocytes, and monocytes, telomere length decreased in 37%, did not change in 52%, and increased in 11% of the participants. Telomerase activity declined with age in resting and activated T cells, and resting but not activated B cells. Percentages of nave T cells decreased and CD28- T cells increased with age throughout the adult life and were positively and negatively correlated to telomere length in T cells, respectively. Finally, a significant portion of the observed telomere attrition with aging was explained by declined telomerase activity, decreased nave cells, increased CD28- T cells, and the presence of specific health conditions (cancer and adiposity). Together, our findings suggest that changes of telomere length, telomerase activity, and composition of subsets in blood lymphocytes in vivo with age are highly coordinated and influenced by health conditions.